Parallel Programming Models and Paradigms

In the 1980s it was believed computer performance was best improved by creating faster and more e cient processors. This idea was challenged by parallel processing, which in essence means linking together two or more computers to jointly solve a computational problem. Since the early 1990s there has been an increasing trend to move away from expensive and specialized proprietary parallel supercomputers (vector-supercomputers and massively parallel processors) towards networks of computers (PCs/Workstations/SMPs). Among the driving forces that have enabled this transition has been the rapid improvement in the availability of commodity highperformance components for PCs/workstations and networks. These technologies are making a network/cluster of computers an appealing vehicle for cost-e ective parallel processing and this is consequently leading to low-cost commodity supercomputing. Scalable computing clusters, ranging from a cluster of (homogeneous or heterogeneous) PCs or workstations, to SMPs, are rapidly becoming the standard platforms for high-performance and large-scale computing. The main attractiveness of such systems is that they are built using a ordable, low-cost, commodity hardware (such